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DETAILED ACTION 

1 . This communication is in response to the Amendment filed 27 March 
2007. 

2. Claims 1-23 and 34 are pending in the current application. Claims 1,8, 
12, 16, 20, 23 and 34 are independent. In the Amendment filed 27 March 2007, 
claims 1-6, 8, 11-13, 16-18, 20, 23 and 34 were amended and claims 24-33 were 
canceled. This action is made Final. 

3. The prior art rejections made of record regarding claims 1-23 and 34 in the 
Office Action dated 22 November 2006 have been withdrawn as necessitated by 
amendment. 



Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

5. Claims 1-23 and 34 rejected under 35 U.S.C. 102(e) as being 



anticipated by US PGPub 2007/0038612 to Sull et al (hereafter Sull). 
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Referring to claim 1, Sull discusses a method of constructing a 
multimedia database method, comprising: 

(a) receiving a start point [starting frame number] and an end point 
[finishing frame number] of each of a plurality of first semantic units [segments] of 
multimedia data, the multimedia data being divided into a plurality of first 
semantic units such that each of the first semantic units can 

be presented to a user upon a keyword-based search as a smallest meaningful 
unit for searching for multimedia data (see [0030] and [0067]); 

(b) receiving at least one keyword [metadata] for each of the first semantic 
units (see [0160]); 

(c) receiving a start point and an end point of at least one second 
semantic unit of the multimedia data, the second semantic unit including at least 
two first semantic units and being associated with keywords of first semantic 
units that are included in the second semantic unit such that the second semantic 
unit can be presented to a user upon the keyword-based search [the metadata of 
segments can form a hierarchical structure where the larger segment contains 
the smaller segments] (see [0030]; [0061]; and [0204]); and 

(d) storing the keyword together with the start point and the end point of its 
corresponding first semantic unit and second semantic unit, wherein the user 
controls a granularity of the keyword-based search result with the same keyword 
by requesting one of the first semantic unit and the second semantic unit be 
presented (see [0166] and [0169]). 
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Referring to claim 2, Suil discloses the method of claim 1 further 
comprising (e) receiving a start point and an end point of at least one third 
semantic unit, the third semantic unit including at least two second semantic units 
and being associated with keywords of second semantic units that are included 
in the third semantic unit such that the third semantic unit can be presented to 
the user upon the keyword-based search, wherein in step (d), a keyword is 
stored with the start point and the end point of its corresponding third semantic 
unit, wherein the user further controls a granularity of the keyword-based search 
result with the same keyword by requesting one of the first semantic unit, the 
second semantic unit and the third semantic unit be presented [As with metafile 
3302, metafile 3304 groups the elements in a hierarchical structure (a and b into 
d, and c and d into e)] . 

Referring to claim 3, Sull discloses the method of claim 1 further 
comprising (f) receiving titles of each of the first semantic units and the second 
semantic unit, wherein in step (d), the keyword is stored with the titles of its 
corresponding first semantic unit and second semantic unit (see [0166] and 
[0167], lines 7-11). 

Referring to claim 4, Sull discloses the method of claim 1 , wherein the 
keyword is classified into one of predetermined categories and is stored together 
with its corresponding category [type of content] in step (d) (see [0300]). 

Referring to claim 5, Sull discloses the method of claim 1 or 4, wherein 
the keyword is classified into a person category [actor], an object category, a 
time category, or a place category (see [0300]). 
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Referring to claim 6, Sull discloses the method of claim 1 further 
comprising (g) storing the keyword together with its similar keywords so that one 
of the first semantic unit and the second semantic unit associated with both the 
keyword and the similar keywords are presented to the user as a search result 
(see [0205]). 

Referring to claim 7, Sull discloses the method of claim 1 , wherein the 
length of each first semantic unit and the length of each second semantic unit are 
determined by a user who constructs the multimedia database (see [0160], lines 
1-4). 

Referring to claim 8, Sull discloses a multimedia database system, 
comprising: 

a multimedia database [multimedia database 310], which stores 
multimedia data (see [0169] and [0310]); 

a keyword database (see [0155]) which stores keywords associated with a 
plurality of first semantic units which are necessary for searching for the 
multimedia data, location information of each of the first semantic units of the 
multimedia data, and location information of at least one second semantic unit of 
the multimedia data, the multimedia data being divided into a plurality of first 
semantic units [segments] such that each of the first semantic units can be 
presented to a user upon a keyword-based search as a smallest unit for 
searching for multimedia data, and the second semantic unit including at least 
two first semantic units and being associated with keywords of first semantic 
units that are included in the second semantic unit such that the second semantic 
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unit can be presented to a user upon the keyword-based search (see [0030]; 
[0061]; and [0204]); 

an input unit which receives the location information of each first semantic 
unit including a start point and an end point of each of the first semantic units, the 
location information of each second semantic unit including a start point and an 
end point of the second semantic unit, and the keywords [metadata] (see [0160]); 
and 

a control unit for controlling the process for storing the keywords with the 
location information of the corresponding first semantic unit and second semantic 
unit and generating a search result with a different granularity of the search result 
with the same keyword depending on a user's input requesting one of the first 
semantic unit and the second semantic unit be presented (see [0166] and 
[0169]). 

Referring to claim 9, Sull discloses the system of claim 8, wherein the 
input unit receives titles of each first semantic unit and each second semantic 
unit, and the control unit stores the titles in the keyword database together with 
their corresponding keywords (see [0166] and [0167], lines 7-11). 

Referring to claim 10, Sull discloses the system of claim 8, wherein the 
input unit receives predetermined categories [type of content] into which the 
keywords are classified, and the controller stores the keywords with their 
corresponding category (see [0300]). 

Referring to claim 1 1 , Sull discloses the system of claim 8, wherein the 
keyword database includes a similar keyword database where keywords having 
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similar meanings are stored, and when a keyword is input via the input unit, the 
controller searches the similar keyword database for a keyword that matches the 
input keyword, and, if there is a search result, stores the input keyword in the 
keyword database together with its similar keywords obtained from the similar 
keyword database so that not only a semantic unit corresponding to the input 
keyword but also semantic units corresponding to its similar keywords can be 
searched for when a search for the semantic unit of the input keyword or any of 
its similar keywords is carried out (see [0204]). 

Referring to claim 12, Sull discusses a method of constructing a 
multimedia database, comprising: 

(a) setting a length of each of a plurality of first semantic units of 
multimedia data, the multimedia data being divided into a plurality of first 
semantic units [segments] such that each of the first semantic units can be 
presented to a user upon a keyword-based search as a smallest unit for 
searching for multimedia data according to a user's input (see [0030] and [0067]); 

(b) extracting a keyword [metadata] from each first semantic unit (see 
[0160]); 

(c) setting a length of at least one second semantic unit of the multimedia 
data, the second semantic unit including at least two first semantic units and 
being associated with keywords of first semantic units that are included in the 
second semantic unit such that the second semantic unit can be presented to a 
user upon the keyword-based search [the metadata of segments can form a 
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hierarchical structure where the larger segment contains the smaller segments] 
(see [0030]; [0061]; and [0204]); and 

(d) storing the extracted keyword with its corresponding first semantic unit 
and second semantic unit, wherein the user controls a granularity of the 
keyword-based search result with the same keyword by requesting one of the 
first semantic unit and the second semantic unit be presented (see [0166] and 
[0169]). 

Referring to claim 13, Sull discloses the method of claim 12, wherein (b) 
comprises: 

(b1 ) extracting voice data from the multimedia data using a speech 
recognition technique (see [0300]); and 

(b2) extracting a predetermined part from the extracted voice data as a 
keyword (see [0300]). 

Referring to claim 14, Sull discloses the method of claim 12, wherein (b) 
further comprises: 

(b3) receiving a first keyword and first keyword information (see [0160]); 

and 

(b4) extracting the first keyword as a keyword of a first semantic unit when 
the first semantic unit has the same keyword information as the received 
keyword information (see [0160]). 

Referring to claim 15, Sull discloses the method of claim 14, wherein the 
keyword information is voice, an image, or text (see [0166]). 
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Referring to claim 16, Sull discloses a multimedia database system, 
comprising: 

a multimedia database [multimedia database 310], which stores 
multimedia data (see [0169] and [0310]); 

a keyword database (see [0155]) which stores keywords associated with a 
plurality of first semantic units [segments] which are necessary for searching for 
the multimedia data, location information of each of the first semantic units of the 
multimedia data and location information of at least one second semantic unit of 
the multimedia data, the multimedia data being divided into a plurality of first 
semantic units such that each of the first semantic units can be presented to a 
user upon a keyword-based search as a smallest unit for searching for 
multimedia data, and the second semantic unit including at least two first 
semantic units and being associated with keywords [metadata] of first semantic 
units that are included in the second semantic unit such that the second semantic 
unit can be presented to a user upon the keyword-based search (see [0030]; 
[0061]; and [0204]); 

a keyword extraction unit which extracts keywords [metadata] from each 
first semantic unit (see [0160]); and 

a control unit for controlling the process for storing the keywords with the 
location information of the corresponding first semantic unit and second semantic 
unit and generating a search result with a different granularity of the search result 
with the same keyword depending on a user's input requesting one of the first 
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semantic unit and the second semantic unit be presented (see [0166] and [0167], 
lines 7-11). 

Referring to claim 17, Sull discloses the system of claim 16 further 
comprising an input unit which receives the location information of each first 
semantic unit, including a start point [starting frame number] and an end point 
[finishing frame number], the location information of each second semantic unit, 
including a start point and an end point, and the keywords (see [0030]; [0166]; 
and [1067], lines 7-11). 

Referring to claim 18, Sull discloses the system of claim 16, wherein the 
keyword extraction unit comprises: 

a voice extractor which extracts voice data from the multimedia data using 
a speech recognition technique (see [0300]); and 

a part-of-speech extractor which extracts a predetermined part of speech 
from the voice data extracted by the voice extractor as a keyword (see [0300]). 

Referring to claim 19, Sull discloses the system of claim 16 further 
comprising an input unit which receives a first keyword and first keyword 
information, wherein the keyword extraction unit extracts the first keyword as a 
keyword of a first semantic unit when the first semantic unit has the same 
keyword information as the received keyword information (see [0160]). 

Referring to claim 20, Sull discloses a method of providing a multimedia 
data search service using a system for providing a multimedia data search 
service, the system including a multimedia database [multimedia database 310] 
which stores multimedia data (see [0169] and [0310]), and a keyword database 
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(see [01 55]) which stores keywords associated with a plurality of first semantic 
units [segments] which are necessary for searching for the multimedia data, 
location information of each first semantic unit of the multimedia data, and 
location information of each second semantic unit of the multimedia data, the 
multimedia data being divided into a plurality of first semantic units such that 
each of the first semantic units can be presented to a user upon a keyword- 
based search as a smallest unit for searching for multimedia data, and the 
second semantic unit including at least two first semantic units and being 
associated with keywords of first semantic units that are included in the second 
semantic unit such that the second semantic unit can be presented to a user 
upon the keyword-based search (see [0030]; [0061]; and [0204]), the method 
comprising: 

(a) receiving a keyword necessary to search for multimedia data (see 
[0169]); 

(b) allowing a user to select a search unit level from a first semantic unit 
and a second semantic unit (see [0169]); 

(c) searching for multimedia data with the keyword in a granularity of the 
search unit level selected by the user (see [0166] and [0169]); and 

(d) outputting a search result to a user, wherein the user controls a 
granularity of the keyword-based search result with the same keyword by 
selecting one of the first semantic unit and the second semantic unit be 
presented (see Fig 28, 61 and 62). 
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Referring to claim 21 , Sull discloses the method of claim 20, wherein 
keywords are stored in the keyword database together with their corresponding 
first and second semantic units' location information and titles, and in (d), titles of 
searched semantic units are displayed on a screen (see [0166] and [0167], lines 
7-11). 

Referring to claim 22, Sull discloses the method of claim 20, wherein in 
(d), the searched semantic units are displayed on a screen together with their 
respective keywords (see Figs 28, 61 and 62). 

Referring to claim 23, Sull discloses a system for providing a multimedia 
data search service, comprising: 

a multimedia database [multimedia database 310] which stores 
multimedia data (see [0169] and [0310]); 

a keyword database (see [0155]) which stores keywords associated with a 
plurality of first semantic units [segments] which are necessary for searching for 
the multimedia data, location information of each first semantic unit of the 
multimedia data, and location information of each second semantic unit of the 
multimedia data, the multimedia data being divided into a plurality of first 
semantic units such that each of the first semantic units can be presented to a 
user upon a keyword-based search as a smallest unit for searching for 
multimedia data, and the second semantic unit including at least two first 
semantic units and being associated with keywords of first semantic units that 
are included in the second semantic unit such that the second semantic unit can 
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be presented to a user upon the keyword-based search (see [0038]; [0061]; and 
[0204]); 

an input unit which receives a keyword [metadata] and a search unit level 
from a user (see [0160]); 

a control unit which searches the keyword database with the keyword 
received from the user, provides links between resulting search results and 
places in the multimedia database where the search results are stored, and 
outputs search results generated with a different granularity of the search result 
depending on a user's input requesting one of the first semantic unit and the 
second semantic unit be presented (see [0166] and [0169]); and 

a display unit [screen] which displays the search results obtained by the 
control unit (see [01 57]). 

Referring to claim 34, Sull discloses a computer-readable recording 
medium on which a program enabling the method of any of claims 1 through 4, 6, 
7, 12 through 15, and 20 through 22 is recorded (see [0159]). 
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Conclusion 

6. Applicant's amendment necessitated the new ground(s) of rejection 
presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. 
See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as 
set forth in 37 GFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 
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